There has been increasing motivation for obtaining visual object data from physical modeling of the image formation process. Most existing vision methods utilize heuristic techniques on image gray values that are based primarily on image properties that ignore the image formation process. These methods are developed largely on a trial and error basis and are made to work in very domain specific environments. These heuristic methods break down very easily upon small deviations from their specifically intended domain.
Vision methods which are primarily based on physical laws are more robust in the sense that their behavior is determined by well defined physical assumptions. The area of physical modeling of image formation pertaining to this invention is called "radiometric modeling". Radiometric modeling usually involves three components; (i) an illumination model, (e.g., incident orientation and incident intensity of light sources), (ii) a reflectance model for objects in the scene (e.g., Lambertian model), and, (iii) camera sensor model that relates gray value representation at a pixel to the actual radiance that is incident at that pixel. Most existing machine vision methods that utilize radiometric modeling compute local surface normal information (i.e., 3-D shape information) on smooth object surfaces. These methods are important particularly for smooth featureless surfaces because stereo triangulation techniques are inadequate due to the lack of features (e.g., edges) for depth computation.
Given a reflectance model for an object surface expressing reflected radiance as a function of surface orientation and other imaging parameters, a sensed reflected radiance value constrains the local surface orientation at the corresponding point to be on a specific locus of surface orientation values. Photometric stereo, when applied to diffuse reflecting surfaces, is a radiometric vision method which disambiguates this surface orientation locus by taking additional sensed reflected radiance values from the same object point for different incident light source orientations. For a simple Lambertian reflectance model surface, only three different non-coplanar incident light source orientations are required to uniquely ascertain local surface orientation. Both point light sources and extended light sources can be utilized.
For highly specular reflecting surfaces, such as metals, implementation of photometric stereo involves the use of extended structured light sources, or a vast array of many different point light sources. This is so as to produce specular reflection from many different possible surface orientations. Determining surface orientation for specular surfaces involves the simple geometry of specular reflection as the reflectance model. Clearly for specular reflection the angle of incidence equals the angle of reflection with respect to the surface normal. If the incident orientation of a lighting element is known relative to the viewing vector of the camera sensor, the normal at the object point specularly reflecting the light received from the lighting element is simply the bisector of the incident light vector and the viewing vector. The major problem to be solved in this implementation of photometric stereo for specular surfaces is the correspondence between specularly reflecting light perceived by the camera sensor at an object point, and the lighting element that produced it. Using and extended light source this is accomplished by placing on the light source multiple, calibrated intensity gradient filters. Multiple images are taken each for a different filter placed on the extended source, and various ratios of sensed specularly reflected radiance from an object point correspond to the calibrated incident orientation of the correct lighting element. Using a vast array of point light sources (e.g., a hemispherical array of thin optic fibers), multiple images are taken while different known subsets of the point light sources are turned on and off. From these multiple images, the on-off sequence for a particular point specular reflection from the object surface uniquely determines the actual point light source from which it was produced. Usually a binary encoding scheme for the light sources is employed to reduce the number of multiple images needed.
In all photometric stereo implementations the camera sensor always remains static between multiple images. Therefore there is no correspondence problem between pixels. However, photometric stereo does require the precise calibration of the incident orientation of multiple light sources, or the calibration of a single light source multiple times as it is moved into different incident orientations. Usually empirical look-up tables for reflectance as a function of surface orientation are computed for each different diffuse surface used. These tables however are very sensitive to typical changes in light source emitted radiance over time. Photometric stereo is generally not applied to "hybrid" regions of an object surface where both diffuse and specular components of reflection are significant. Another big limitation of photometric stereo is that the camera sensor is restricted to an orthographic field of view. Afterall, the calibration of the incident orientation of a light source is only applicable to a small region of space that an object can occupy. The incident orientation for a given light source may be extremely variable throughout a wide perspective field of view unless the light source is very far away.
Some limited results have been reported using radiometric modeling to separate out diffuse and specular reflection components, based on color analysis. This work is limited to inhomogeneous dielectrics such as plastics and rubber. On inhomogeneous dielectrics the specular component of reflection is the same color as the illuminating light source. If this color is distinct from the intrinsic color of the inhomogeneous dielectric (i.e., the color of the diffuse component) then color analysis can quantitatively separate out the two reflection components. If the color of the light source is unknown, heuristic segmentation procedures need to be invoked to determine the color of the light source.
The only known use of polarizing filters in machine vision is to visually suppress strong specular regions using the method of "cross polorization". Cross polarization is a method whereby a linear polarizing filter is placed over the illuminating light source as well as over the camera sensor. The orientations of these two polarizers are 90.degree. with respect to one another so that the reflecting specular glare off of an object surface gets canceled out. The use of polarizers in machine vision has only been known to improve image quality rather than used to extract physical information from an object scene.
No machine vision method is known whatsoever that can classify an object surface as being metal or dielectric.